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EXECUTIVE  SUMMARY 


The  Aviation  Research  Group  (ARG)  in  the  De¬ 
partment  of  Psychology  at  New  Mexico  State  Uni¬ 
versity  (NMSU)  has  research  interests  relating  to 
several  aspects  of  the  organization  and  display  of 
information  in  the  cockpit.  In  this  report,  we  present 
a  systematic  analysis  of  the  information  required  by 
pilots  in  various  phases  of  flight.  Our  aim  is  to  specify 
what  information  is  needed,  when  it  is  needed,  and 
how  pilots  conceive  of  the  organization  of  the  infor¬ 
mation.  This  project  was  devoted  to  an  analysis  of  the 
information  required  in  normal  flight  operations  as 
contrasted  with  unusual  or  emergency  situations.  In 
future  work,  we  plan  to  conduct  a  similar  study  of 
information  requirements  in  various  emergencies. 

The  nature  of  the  present  study  is  largely  descriptive 
in  that  it  provides  numerous  analyses  of  the  priorities 
pilots  assign  to  different  information  elements.  The 
study  required  the  development  of  various  taxonomies 
including:  (a)  Information  used  in  flight  and  planning, 
(b)  Phases  of  flight,  and  (c)  Sources  of  information  used 
in  flight.  The  analyses  show  overall  priorities  of  the 
information  elements  in  various  phases  of  flight  and 
how  these  priorities  compare  for  more-  versus  less- 
experienced  pilots.  Not  surprisingly,  priorities  change 
over  the  course  of  a  flight,  and  we  proposed  that  the 
design  of  new  displays  would  benefit  from  considering 
these  changing  priorities.  We  also  present  data  on  how 
pilots  conceive  of  the  organization  of  the  various  ele¬ 
ments  of  information.  These  data,  too,  can  assist  display 
designers  in  layout  of  information. 

The  differences  in  information  priorities  as  a  func¬ 
tion  of  flying  experience  identify  some  issues  that 
flight  instructors  should  consider  in  flight  training. 
Overall,  more  experienced  pilots  tend  to  give  higher 
priorities  to  several  information  elements  as  though 
they  are  considering  more  factors  with  higher  prior¬ 
ity.  In  particular  phases  of  flight,  less  experienced 


pilots  give  lower  priority  to  communications  both 
with  ATC  and  with  traffic  than  do  more  experienced 
pilots.  The  less  experienced  pilots  assign  higher  pri¬ 
ority  to  vertical  velocity  than  do  more  experienced 
pilots.  A  discussion  of  these  differences  and  others 
can  be  found  in  the  report. 

Our  analysis  of  the  sources  of  information  re¬ 
quired  in  flight  shows  the  dramatic  impact  of  GPS 
technology  on  aviation.  The  analysis  of  sources  also 
allows  a  determination  of  the  degree  to  which  redun¬ 
dant  information  is  available.  Clearly,  safety  may  be 
compromised  when  information  is  only  available 
from  a  single  source. 

Modern  aviation  is  undergoing  several  dramatic 
changes  that  affect  nearly  every  aspect  of  aircraft 
operating  in  the  national  airspace  system.  Among 
these  changes  are:  (a)  the  move  to  “Free  Flight,”  (b) 
the  development  of  new  control  systems  for  aircraft, 
(c)  development  of  new  information  displays  and 
methods  for  selecting  displays,  and  (d)  changes  in 
processes  and  policies  of  aviation-related  organiza¬ 
tions.  As  these  changes  come  about  and  as  an  aid  in 
planning,  it  is  valuable  to  have  models  that  can  help 
analyze  the  effect  of  the  changes  on  critical  issues  of 
safety,  automation,  workload,  and  situation  aware¬ 
ness.  We  have  presented  a  pilot-centered  model  of 
information  requirements  in  flight  tasks  emphasiz¬ 
ing  the  source,  priority,  and  mental  organization  of 
the  information  elements.  This  model  should  allow 
one  to  make  informed  decisions  about  the  design  of 
new  technology  including  information  displays.  The 
model  should  also  be  useful  for  evaluating  the  effect 
of  changes  in  the  airspace  system  by  providing  baseline 
information  about  where  pilots  can  obtain  informa¬ 
tion  in  the  current  system  and  by  reminding  policy 
makers  about  what  information  pilots  need  and  when 
they  need  it. 
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Priorities,  Organization,  and  Sources  of  Information 
Accessed  by  Pilots  in  Various  Phases  of  Flight 


introduction 

This  report  summarizes  the  research  accomplish- 
ments  for  a  one-year  project  investigating  the  priori¬ 
ties  pilots  assign  to  different  information  in  various 
phases  of  flight  under  normal  operating  conditions. 
We  use  the  term,  information,  to  emphasize  our  focus 
on  interpreted  data,  in  contrast  to  raw  data.  For 
example,  when  we  refer  to  information  about  Traffic, 
we  mean  the  knowledge  that  aircraft  traffic  is  present 
in  some  particular  position  rather  than  the  particular 
data  that  leads  to  the  knowledge.  Thus,  our  analysis 
is  directed  at  a  cognitive  level  in  contrast  to  a  concern 
with  perceptual  processes.  The  term,  information 
elements,  is  used  to  refer  to  the  knowledge  pilots  come 
to  possess  (or  should  possess)  during  the  course  of 
flight.  An  understanding  of  how  pilots  prioritize  and 
organize  flight-related  information  can  be  of  value  in 
the  design  of  display  systems  for  pilots.  The  data 
presented  in  this  report  can  also  help  characterize 
what  information  pilots  need  as  we  attempt  to  char¬ 
acterize  the  nature  of  the  National  Airspace  System. 
This  introductory  section  will  provide  the  rationale 
for  the  research  effort,  as  well  as  background  for  the 
research. 

Motivation 

There  were  several  motivations  for  us  to  undertake 
the  research  reported  here.  First,  a  characterization  of 
information  priorities  related  to  various  phases  of 
flight  together  with  an  analysis  of  the  mental  organi¬ 
zation  associated  with  the  information  would  be 
useful  in  the  design  of  new  displays  for  aircraft.  Those 
displays  could  then  be  organized  to  present  only  the 
most  salient  information  for  any  given  phase  of  flight 
and  in  a  well-integrated  fashion,  reduce  the  cognitive 
requirements,  and  thus  the  workload,  imposed  on  the 
pilot.  The  end  result  should  be  that  the  pilot  can 
devote  more  effort  to  other  tasks  not  directly  associ¬ 
ated  with  extracting  information  from  cockpit  dis¬ 
plays.  Second,  this  approach  may  provide  alternative 
to  traditional  task  analysis  by  examining  operator 
perceptions  and  cognitive  organizations  of  data. 


Third,  identifying  the  priorities  of  information  should 
allow  us  to  characterize  how  pilots  view  information 
needs  in  the  current  National  Airspace  System,  and  to 
derive  from  that  how  changes  in  the  use  of  that 
airspace  or  the  allocation  of  tasks  between  actors  in 
the  system  may  change  perceived  priorities.  This 
should  then  have  an  influence  on  how  future  displays 
are  designed  to  best  support  any  new  tasks  or  tasks 
performed  in  an  altered  airspace  environment. 

Display  Design 

As  modern  aviation  adopts  new  technology  for  the 
display  of  information  and  the  control  of  aircraft 
systems,  there  is  a  clear  shift  away  from  individual 
dedicated  displays  (e.g.,  altimeter,  airspeed  indica¬ 
tor,  turn-and-bank  indicator)  toward  integrated  dis¬ 
plays  in  which  most  information  comes  from  a 
common  source  such  as  panel-mounted  cathode-ray 
tubes  (CRT’s),  head-up  displays  (HUDs),  or  helmet- 
mounted  displays  (HMDs).  Some  critical  issues  in 
the  design  and  engineering  of  such  displays  concern 
the  decisions  about  what  information  to  display  at 
any  given  time,  as  well  as  how  to  display  it. 

Design  decisions  about  what  to  display  at  what 
times  would  benefit  considerably  from  a  thorough 
analysis  of  the  various  required  elements  of  informa¬ 
tion  (aircraft  position,  weather,  engine  status,  com¬ 
munications,  etc.)  and  their  properties  (priority  of 
the  information,  source  of  the  information,  quality 
of  the  information,  etc.)  in  various  phases  of  flight. 
Together  with  data  revealing  how  pilots  organize 
these  information  elements,  the  analysis  can  assist 
efforts  toward  integration  in  displays  and  prior¬ 
itization  and  layering  of  information  elements.  For 
example,  displays  designed  so  that  critical  informa¬ 
tion  is  available  when  needed  in  a  practical  format 
should  be  of  great  assistance  to  the  pilot  who  is 
actively  controlling  the  aircraft.  Andre  and  Wickens 
(1991)  demonstrated  the  ability  of  an  integrated 
three-dimensional  cockpit  display  system  to  reduce 
mental  workload  while  increasing  situation  aware¬ 
ness.  Well-designed  displays  should  also  help  estab¬ 
lish  and  maintain  the  pilot’s  situational  awareness 
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when  automated  systems  are  in  control.  Research  on 
mode  errors  and  mode  awareness  in  supervisory  con¬ 
trol  (Sarter  &  Woods,  1992)  indicates  that  the  in¬ 
creased  capabilities  and  autonomy  of  new  automated 
systems  have  made  mode  awareness  problems  more 
prevalent  in  the  modern  cockpit. 

Comparison  With  Task  Analysis 

A  long-standing  goal  in  the  analysis  of  aircraft 
systems  is  to  produce  thorough  analyses  of  the  tasks 
involved  in  flying  aircraft  and  the  information  re¬ 
quired  to  perform  the  tasks  (Alexander  C.  Williams, 
Jr.  in  Roscoe,  1980).  Although  the  analysis  we  pro¬ 
pose  to  perform  is  related  to  task  analysis,  it  is  also 
distinct  in  some  important  ways.  In  the  usual  ap¬ 
proaches  to  task  analysis,  goal-directed  activities  are 
analyzed  in  terms  of  goals  to  be  accomplished  and  the 
actions  required  to  accomplish  those  goals  (cf. 
Sutcliffe,  1997).  A  limitation  of  this  approach  in  the 
context  of  aviation  is  that  the  actions  required  of  a 
pilot  to  accomplish  goals  are  largely  determined  by 
the  nature  of  the  technology  in  the  aircraft  so,  in 
essence,  every  aircraft  calls  for  a  distinct  task  analysis. 
Of  course,  by  abstracting  the  actions  somewhat,  one 
could  reduce  the  number  of  distinct  task  analyses 
required,  but  abstracting  would  often  move  the  analy¬ 
sis  from  the  task  per  se  toward  the  goals  of  the  tasks. 
For  example,  we  could  use  “put  in  1 0  degrees  of  flaps” 
instead  of  “move  the  flap  control  to  the  1 0  degrees 
mark,”  but  in  doing  so,  we  have  technically  moved 
from  specifying  an  action  to  specifying  a  goal  (or, 
better,  a  subgoal).  It  seems  clear  that  goals  do  remain 
more  constant  across  technological  differences  than 
actions  do.  Even  with  extreme  degrees  of  automation 
in  aircraft,  most  goals  of  flight  remain  the  same  as 
they  were  with  pilots  in  control  throughout  the 
flight^  Still,  we  should  be  able  to  provide  a  more 
thorough  analysis  of  the  various  phases  of  flight  than 
simply  specifying  the  goals  (and  subgoals)  of  each  phase 
and  still  have  some  generality  across  technologies. 

Our  approach  assumes  that  goals  remain  relatively 
constant  across  technological  changes,  and  the  criti¬ 
cal  information  pertaining  to  flight  also  remains 
relatively  constant.  For  the  pilot  actively  controlling 
the  aircraft,  it  is  obvious  that  particular  information 
elements  are  critically  important  at  different  times  in 
a  flight,  but  it  is  also  true  that  even  when  the  aircraft 
is  being  flown  by  automated  systems,  the  same 


information  elements  are  critical.  Moreover,  for  pi¬ 
lots  to  maintain  (or  to  quickly  attain)  good  situ¬ 
ational  awareness,  they  must  have  these  critical 
information  elements  whether  they  are  actively  in 
control  or  not.  We  also  propose  that  pilots’  mental 
models  of  flight  depend  upon  having  the  appropriate 
information  elements.  When  the  pilot’s  model  contains 
incorrect  information,  “pilot  error”  is  not  far  behind. 

Evaluating  Changes  in  the 
National  Airspace  System 
Another  application  of  the  data  we  report  is  found 
in  its  potential  for  assisting  in  the  analysis  of  the 
consequences  of  changes  in  the  National  Airspace 
System.  Our  analysis  identifies  the  critical  informa¬ 
tion  elements  for  the  various  phases  of  flight.  Given 
these  data,  it  is  reasonable  to  ask  about  how  access  to 
this  critical  information  will  be  affected  by  any  change 
in  the  system.  If  a  proposed  change  in  the  system 
eliminates  certain  information,  our  analysis  should 
help  to  determine  whether  that  information  is  usually 
used  and,  if  so,  alternative  sources  of  the  information 
could  be  sought.  Studies  could  then  determine 
whether  the  alternatives  are  adequate  or  whether  new 
options  should  be  considered.  In  summary,  we  see 
our  model  as  providing  baseline  information  about 
the  current  state  of  the  National  Airspace  System  that 
will  be  useful  in  planning. 

General  Plan  of  the  Investigation 
The  research  team  consisted  of  several  pilots  and 
student  assistants  in  the  Aviation  Research  Group 
(ARG)  at  New  Mexico  State  University,  including 
two  pilot  consultants.  The  decisions  about  materials 
and  taxonomies  were  made  by  consulting  with  the 
pilots  on  the  research  team.  The  research  involved 
three  distinct  projects: 

Project  1.  Selecting  particular  phases  of  flight, 
developing  a  taxonomy  of  information  elements  ac¬ 
cessed  during  these  phases,  developing  a  taxonomy  of 
information  sources,  and  identifying  the  sources  of 
each  information  element. 

Project  2.  Collecting  priority  ratings  from  pilots 
for  each  information  element  in  each  phase  of  flight. 

Project  3.  Collecting  relatedness  ratings  from  pi¬ 
lots  for  a  subset  of  the  information  elements  to  allow 
for  an  analysis  of  the  organization  of  the  information 
elements. 


*  There  are  clear  exceptions  where  certain  goals  cannot  be  preserved  with  automation  as  with  the  goal  of  realizing  the  pleasure  of 
flying  well. 
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Project  1:  Phases  of  Flight,  Information 
Elements,  and  Information  Sources 

The  ARG  team  developed  initial  taxonomies  of 
ph  ases  of  flight,  the  information  elements  accessed  in 
these  phases,  and  the  sources  of  these  information 
elements  in  the  National  Airspace  System.  The  tax¬ 
onomies  were  refined  by  discussions  with  pilots  re¬ 
sulting  in  the  final  products. 

The  phases  of  flight  are  shown  in  Table  1 .  To  make 
the  project  manageable  within  the  one-year  period 
for  the  work,  we  initially  decided  to  restrict  our  work 
to  flying  phases,  as  contrasted  with  ground  phases. 
The  major  concerns  of  safety  lie  in  the  flying  phases. 
However,  it  became  clear  that  it  was  also  necessary  to 
include  some  planning  phases,  as  well  as  flying  phases, 
because  preliminary  testing  suggested  that  pilots  want 
to  include  considerations  of  planning.  If  separate 
planning  phases  are  not  specified,  considerations  of 
planning  will  be  included  in  flying  phases.  To  keep 
planning  and  flying  distinct,  it  was  decided  to  in¬ 
clude  two  planning  phases.  Preflight  and  In-Flight 
Planning,  in  addition  to  flying  phases.  In-Flight 
Planning  was  included  to  keep  needed  planning  in¬ 
formation  distinct  from  information  used  in  flying, 
particularly  in  the  Cruise  Phase. 

The  flying  phases  we  included  are  those  found  in 
most  discussions  of  the  phases  of  flight.  The  Transi¬ 
tion  to  Cruise  Phase  was  included  to  keep  the  infor¬ 
mation  used  there  distinct  from  the  Climb  and  the 


Cruise  Phases.  In  contrast,  the  Transition  to  Descent 
was  not  included  because  our  pilot  consultants 
thought  that  the  transition  was  not  that  distinct  from 
the  Descent  itself.  Finally,  the  analysis  was  restricted 
to  normal  operations,  as  distinct  from  emergencies, 
to  keep  the  project  manageable  in  the  time  frame.  A 
follow-on  study  will  conduct  a  similar  analysis  of 
various  emergencies. 

Table  2  was  arrived  at  after  performing  a  prelimi¬ 
nary  analysis  of  the  information  required  in  normal 
flight  operations.  To  keep  the  number  of  elements 
manageable,  some  of  the  items  represent  large  catego¬ 
ries  of  elements  (e.g..  Aircraft  Configuration,  Engine 
Health,  Airport  Configuration,  General  Weather). 
Other  elements  refer  to  very  specific  items,  such  as 
Pitch,  Bank,  Airspeed,  and  Altitude.  The  choice  of 
these  elements  is  intended  to  be  appropriate  for 
evaluating  information  priorities  in  the  various  phases 
of  flight. 

Sources  of  Information  in  the  Current 
Airspace  System 

To  help  relate  our  analyses  of  information  require¬ 
ments  to  the  state  of  the  National  Airspace  System, 
we  also  undertook  an  analysis  of  how  pilots  obtain 
the  information  they  need  in  flight.  First  the  tax¬ 
onomy  of  information  sources  shown  in  Table  3  was 
developed.  The  sources  distinguish  major  categories  of 
information  sources  (Communications,  Documents, 


Table  1.  Phases  of  flight. 

Percent  of  Percent  of 
Phases  of  Flight _ Flight  Time _ Accidents^ 


Pre-flight  Planning 

NA 

NA 

Takeoff 

2% 

21.4% 

Climb 

13% 

8.8% 

Transition  to  Cruise 

(no  data) 

(no  data) 

Cruise 

60% 

8.8% 

In-flight  Planning 

NA 

NA 

Descent 

10% 

13.4% 

Approach 

14% 

41.8% 

Landing 

1% 

3.8% 

^  The  statistics  on  percent  of  flight  time  and  percent  of  accidents  come  from  a 
1985  report  by  Boeing  reported  in  Nagel  (1988).  This  report  distinguished 
between  Initial  Climb  and  Climb.  Percentages  shown  combine  Initial  Climb 
with  Takeoff.  The  report  also  distinguished  between  Initial  Approach  and  Final 
Approach,  whereas  percents^es  shown  combine  these  into  Approach.  The 
statistics  may  be  more  representative  of  air  carrier  aviation  than  of  general 
aviation. 
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Table  2,  Taxonomy  of  29  information  elements. 


Aircraft 

Configuration 

Aircraft  configuration 
Engine 

Engine  health 
Fuel  quantity 
Fuel  selection 
RPM  (power) 
Position 

Altitude  -  AGL 
Altitude  -  MSL 
Distance 
Orientation 
Bank 

Pitch  (attitude) 

Yaw 

Direction 

Course 

Heading 

Track 

Waypoints 

Speed 

Airspeed 
Ground  speed 
Vertical  velocity 
Schedule 

Time  -  ETA/ETE 
Environment 
Airport 

Airport  configuration 
Runway  aim  point 
Runway  remaining 
Communications 
ATC  Comm 
Traffic  /  other  Comm 
Hazards 

Obstructions 

Traffic 

Regulations 

Airspace 

Weather 

General 

Wind 


Expanded  Description  of  each  Element 

Flaps,  landing  gear,  cowl  flaps,  speed  brakes,  spoilers 

General  health  of  engines 
Fuel  remaining 

Current  selected  source  of  fuel 
Power  setting  or  desired  power 

Altitude  above  ground  level 
Altitude  above  Mean  Sea  Level 
Distance  to  waypoint  or  airport 

Bank 

Pitch 

Yaw 

Desired  track  -  Planned  course 
Nose  direction 
Actual  track  over  the  ground 
Location  and  type  of  enroute  waypoints 

Indicated  airspeed 
Actual  speed  over  the  ground 
Climb  rate  or  descent  rate 

Estimated  time  of  arrival  /  Estimated  time  enroute 


Altitude,  runways,  approaches,  NOT  AMs,  active  runway 

Desired  point  of  touchdown 

Length  of  remaining  runway  on  takeoff  or  landing 

ATC  instructions,  clearances,  etc. 

Communication  with  traffic  or  other  information  source 

Towers,  trees,  mountains,  etc. 

Other  aircraft 

Type  of  airspace  A  B  C  D,  restricted,  MOA  etc. 

General  weather  conditions 
Wind  direction  and  intensity 
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Table  3.  Taxonomy  of  sources  of  information 
needed  in  flight. 

Communications 

ATC  Air  Traffic  Control  Communications 
Comm  Other  (Flight  Service,  Traffic,  ATIS,  etc.) 


Documents 

Chart 

Charts 

ffist 

History  (Aircraft  or  Flight) 

Plan 

Fhght  Plan 

Instruments 

ASI 

Airspeed  Indicator 

AI 

Attitude  Indicator  (Artificial  Horizon) 

ALT 

Altimeter 

TC 

Turn  Coordinator 

HI 

Heading  Indicator 

VSI 

Vertical  Speed  Indicator 

Engl. 

Engine  &  Fuel  (RPM,  Oil  Pressure, 

Temperature,  etc.) 

Clock 

Clock 

DME 

Distance  Measuring  Equipment 

GPS 

Global  Positioning  System 

VOR 

VHF  Omnidirectional  Radio  Range 

Direct  Perception 

Aud  Auditory  (Excluding  Communications) 

Vis  Visual  (Excluding  Reading  Instruments) 


Instruments,  and  Direct  Perception).  For  each  of  the 
information  elements  (Table  2),  we  identified  the 
sources  that  can  provide  the  information.  These 
resulting  sources  for  each  information  element  are 
shown  in  Table  4.  The  sources  of  information  are 
distinguished  according  to  whether  the  information 
is  directly  available  (D),  available  through  inference 
using  multiple  sources  of  information  and/or  as¬ 
sumptions  (I),  or  is  planning  information  producing 
desired  or  expected  values  (P).  Clearly,  direct  infor¬ 
mation  is  more  readily  available  and  immediately 
usable  than  is  information  that  must  be  inferred.  Thus, 
one  view  of  Table  4  might  emphasize  the  “D”  entries. 

Several  observations  can  be  made  about  the  out¬ 
come  of  the  analysis  of  information  sources  shown  in 
Table  4.  First,  the  observation  that  much  of  the 
information  is  available  from  direct  visual  perception 
(not  counting  reading  instruments)  brings  home  the 
point  that,  without  visual  conditions,  flight  is  a 
strikingly  different  activity.  The  potential  differences 
between  instrument  meteorological  conditions  (IMC) 
and  visual  meteorological  conditions  (VMC)  are  also 
dramatized  by  the  number  of  different  information 


elements  available  from  ATC.  The  table  also  reveals 
redundancy  in  information  sources.  Safety  is  cer¬ 
tainly  enhanced  by  the  availability  of  alternate  sources 
of  information  in  case  one  source  should  fail.  Finally, 
the  ubiquity  of  GPS  as  an  information  source  sug¬ 
gests  that  this  technology  is  destined  to  become  a 
major  resource  in  the  future  if  it  hasn’t  already.  As 
others  in  the  human  factors  and  pilot  communities 
have  noted,  there  are  problems  of  over  reliance,  over 
confidence,  and  lack  of  familiarity  with  particular 
GPS  units  that  can  lead  to  dangerous  situations.  At 
the  same  time,  the  value  of  the  information  provided 
by  GPS  systems  can  greatly  simplify  tasks  of  navigat¬ 
ing  leading  to  lower  workload.  Making  the  system 
work  well  and  making  it  reliable  and  usable  are 
challenges  for  pilots,  manufacturers,  and  researchers. 

Project  2:  The  Priorities  of  the  Information 
Elements  in  Various  Phases  of  Flight 

During  development  of  the  priority-rating  task, 
we  considered  distinguishing  between  visual  and 
instrument  flight  for  some  of  the  properties  of  infor¬ 
mation  elements  because  the  sources  of  information 
often  depend  on  this  distinction.  Our  initial  tests, 
however,  indicated  that  the  differences  were  minor, 
so  we  simply  asked  for  general  priorities. 

Following  the  selection  of  the  phases  of  flight  and 
the  development  of  the  taxonomy  of  information 
elements,  we  began  to  collect  the  priority  rating  data 
by  distributing  questionnaires  at  several  sites  around 
the  country.  The  details  of  this  methodology  are 
provided  in  the  following  sections. 

METHOD 

Participants 

Twenty-seven  certified  pilots  of  varying  age  and 
experience  were  used  as  sources  for  the  priority  rating 
data.  To  examine  the  effect  of  experience  on  judg¬ 
ments  of  priorities,  we  separated  the  pilots  into  two 
groups  based  on  their  total  flying  time.  The  hours  of 
the  1 1  less  experienced  pilots  (novices)  ranged  from 
65  to  820  with  a  mean  of  361  and  a  standard  devia¬ 
tion  of 225.  The  16  more  experienced  pilots  (experi¬ 
enced)  had  a  range  of  flying  time  from  1,600  to 
17,000  hours  with  a  mean  of  6,634  and  a  standard 
deviation  of 4,352.  Fifty-four  percent  of  the  less  expe¬ 
rienced  pilots  held  the  Instrument  Rating,  while  88%  of 
the  more  experienced  pilots  were  instrument  rated. 
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Table  4.  Sources  of  the  information  elements  in  the  National  Airspace  System. 


Note.  D  =  Directly  obtained  (current)  values 
I  =  Inferred  (current)  values  using  multiple  sources 
P  =  Planned  or  expected  values 
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Materials 

The  priority  rating  data  were  collected  using  a 
questionnaire  that  included  demographic  informa¬ 
tion  (Appendix  A),  a  definition  of  the  information 
elements  (Table  2),  instructions  for  the  priority  rat¬ 
ings  (Appendix  B),  and  a  form  listing  the  information 
elements  on  the  rows,  the  phases  of  flight  on  the 
columns  and  cells  for  assigning  the  priority  ratings 
for  each  information  element  in  each  phase  of  flight 
(Appendix  B).  The  definition  of  the  priority  ratings 
is  shown  in  Table  5.  The  cells  left  blank  were  coded 
as  4  (lowest  priority)  for  analysis  of  the  priority 
scores. 


Table  5.  Definition  of  the  priority  ratings. 

Priority  Description 

1  Critical  and/or  Frequently  Accessed 

2  Important  and/or  Usually  Accessed 

3  Relevant  and/or  Sometimes  Accessed 

blank  Not  Relevant  or  Rarely  Accessed 

Note.  Blanks  were  coded  as  4  for  analysis. 


Design  and  Procedure 

Collection  of  the  priority  data  occurred  over  an 
eight-month  period.  The  participants  were  given  the 
priority-rating  packet  and  a  stamped,  addressed  en¬ 
velope.  Upon  completion,  the  participants  returned 
the  demographic  information  and  the  priority  ratings 
through  the  mail. 

RESULTS  AND  DISCUSSION 

Overall  Priorities  for  Phases  of  Flight  and 
Information  Elements 

The  priority  rating  data  lends  itself  to  several 
different  analyses.  One  way  to  examine  these  ratings 
is  to  average  them  across  information  sources  within 
a  ph  ase  of  flight,  the  result  reflecting  how  many 
sources  of  information  were  rated  as  being  critical 
during  that  phase.  The  average  would  approach  1  as 
more  sources  were  rated  “critical.”  The  average  prior¬ 
ity  ratings  for  each  phase  of  flight  are  shown  in  Figure  1 
By  way  of  comparison,  the  mean  priority  ratings 
were  2.45  and  2.35  for  Preflight  Planning  and  In¬ 
flight  Planning,  respectively.  For  the  flying  phases, 


Phase  of  Flight 

Tran  to 

Takeoff  Climb  Cruise  Cruise  Descent  ^proach  Landing 


Figure  1.  Mean  priority  for  flying  phases. 


^  This  figure  and  several  others  have  lines  conneaing  the  points.  Strictly  speaking,  the  lines  are  inappropriate  because  the  abscissa 
represents  discrete  points  rather  than  a  continuum.  However,  the  addition  of  lines  facilitates  reading  the  figures.  Think  of  the  lines  as 
leading  the  eye  from  point  to  point,  rather  than  as  an  interpolation  between  points. 
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the  highest  average  priority  ratings  were  for  the  Ap¬ 
proach,  followed  by  the  Landing,  Takeoff,  Descent, 
Climb,  Cruise,  and  Transition  to  Cruise.  Climb  and 
Descent  had  nearly  identical  means.  This  ordering  of 
priority  ratings  appears  to  roughly  correspond  to 
what  pilots  often  report  as  the  relative  difficulty  of 
the  different  phases  of  flight.  In  a  later  section,  we 
discuss  the  relationship  between  priorities  and 
workload. 

It  is  also  of  interest  to  examine  how  the  priority 
ratings  were  distributed  across  the  four  categories  (1 
=  top  priority  to  4  =  irrelevant  or  rarely  accessed) .  The 
distributions  are  shown  in  Figure  2.  The  figure  shows 
the  proportion  of  each  of  the  4  rating  categories  in 
each  phase  of  flight.  The  proportions  sum  to  1  for 
each  phase.  These  distributions  show  that  the  overall 
higher  average  priorities  in  the  more  demanding 
phases  of  flight  are  primarily  due  to  an  increase  in  the 
proportion  of  information  elements  receiving  the 
highest  priority  rating.  These  data  also  suggest  that 
the  3'"*  category  (Relevant  and/or  Sometimes  Ac¬ 
cessed)  is  not  used  very  often.  However,  its  frequency 


does  increase  in  the  less  demanding  phases  of  flight 
where,  presumably,  pilots  are  monitoring  several 
factors  on  an  occasional  schedule. 

In  addition  to  looking  at  priorities  for  phases  of 
flight,  it  is  informative  to  see  how  priorities  are 
assigned  to  the  information  elements  overall.  The 
overall  averages  are  shown  in  Table  6  with  separate 
columns  for  the  flying  phases  and  the  planning  phases. 
There  are  clear  differences  in  the  priorities  associated 
with  planning  and  with  flying.  Of  course,  we  need  to 
examine  the  information  priorities  separately  for  each 
of  the  phases,  but  the  overall  priorities  give  a  general 
idea  of  which  items  of  information  are  of  interest 
throughout  a  flight  (those  at  the  top  of  the  list).  The 
planning  priorities  reflect  the  relative  importance  of 
the  different  information  elements.  Proper  planning 
should  include  special  consideration  of  the  high- 
priority  items.  Items  at  the  end  of  the  list  may  be 
important  in  particular  phases  but  not  overall.  For 
example.  Runway  Remaining  has  an  average  priority 
near  1  in  Landing,  in  contrast  to  near  4  in  Cruise. 


Phases  of  Flight 

Figure  2.  Distribution  of  ratings  across  phases  of  flight. 
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Table  6.  Average  priorities  of  information  elements. 


Flying 

Planning 

Information  Element 

Priority 

Information  Element 

Priority 

RPM  (power) 

1.44 

Fuel  quantity 

1.35 

Airspeed 

1.49 

General  weather 

1.39 

Engine  health 

1.58 

Time  -  ETA/ETE 

1.57 

Traffic 

Wind 

1.61 

Pitch  (attitude) 

Airport  configuration 

1.78 

General  weather 

1.79 

Waypoints 

1.85 

ATC 

1.79 

Altitude  -  MSL 

1.87 

Aircraft  configuration 

1.80 

Distance 

1.93 

Altitude  -  MSL 

1.85 

Airspace 

1.96 

Heading 

1.89 

Course 

2.07 

Altitude  -  AGL 

1.92 

Obstructions 

2.07 

Wind 

1.98 

Ground  speed 

2.09 

Traffic  /  other  Comm 

1.99 

Engine  health 

2.31 

Course 

2.13 

ATC 

2.35 

Obstructions 

2.13 

Fuel  selection 

2.35 

Track 

2.13 

Heading 

2.39 

Airspace 

2.19 

RPM  (power) 

2.44 

Vertical  Velocity 

2.20 

Altitude  ”  AGL 

2.44 

Bank 

2.24 

Airspeed 

2.50 

Fuel  selection 

2.24 

Traffic  /  other  Comm 

2.57 

Yaw 

2.25 

Track 

2.67 

Fuel  quantity 

2.30 

Aircraft  configuration 

2.67 

Airport  configuration 

2.55 

Traffic 

2.74 

Distance 

2.65 

Runway  remaining 

3.22 

Ground  speed 

2.74 

Bank 

3.43 

Waypoints 

2.76 

Runway  aim  point 

2.97 

Vertical  Velocity 

3.48 

Time  -  ETA/ETE 

2.98 

Yaw 

3.50 

Runway  remaining 

2.99 

Runway  aim  point 

3.54 
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Priority  and  Pilot  Experience  Level 
It  is  of  interest  to  consider  how  pilot  experience 
relates  to  the  judgment  of  priorities.  The  differences 
as  a  function  of  experience  were  small,  but  there  are 
some  differences  worth  considering.  Average  priority 
ratings  for  the  various  phases  of  flight  and  pilot 
experience  level  are  shown  in  Figure  3.  Values  for 
planning  are  shown  at  the  right  end  of  the  graph  for 
comparison.  The  more  experienced  pilots  assigned 
higher  overall  priorities  for  the  most  demanding 
phases  of  flight.  The  0.12  overall  difference  in  the 
average  priority  is  significant,  t{G)  -  2.62, =  0.031. 

Information  Priorities  in  Phases  of  Flight 
Next,  we  turn  to  an  analysis  of  the  details  of  the 
priorities  in  each  phase  of  flight.  The  goal  of  this 
analysis  is  to  define  which  information  elements  have 
high  priority  in  particular  phases.  Such  data  would  be 
useful  in  designing  dynamic  information  displays 
that  provide  the  most  relevant  information  at  the 
proper  time.  Figure  4  through  Figure  12  show  the 
information  priorities  associated  with  each  phase  of 
flight.  The  figures  are  organized  so  that  the  informa¬ 
tion  elements  are  sorted  for  each  phase  of  flight  using 
the  average  priority  of  the  more-experienced  pilots 
(Exp).  The  average  priority  of  the  novice  pilots  (Nov) 
is  also  shown  for  comparison.  Points  of  substantial 


differences  between  the  two  groups  of  pilots  may 
identify  areas  that  could  usefully  be  addressed  in 
training.  Table  7  lists  the  significant  differences  us¬ 
ing  p  =  0.05.  All  differences  with  chance  probability 
less  than  or  equal  to  0.05  are  shown  along  with  the 
probability  of  a  difference  at  least  this  large  by  chance 
according  to  a  two-tailed  r-test  with  25  df.  Because 
these  differences  are  selected  from  a  large  set  of  tests 
(9  ph  ases  of  flight  x  29  information  elements  =  261 
tests),  we  would  expect  to  find  13  (5%  of  261)  such 
significant  differences  by  chance  alone.  Thus,  the  16 
significant  differences  found  should  be  viewed  with 
caution.  The  lower p  values  shown  on  the  table  show 
which  differences  are  most  likely  to  be  reliable,  and 
the  repeated  occurrence  of  significant  differences 
associated  with  the  same  information  element  leads 
to  greater  confidence  in  those  effects. 

As  the  overall  means  suggested  earlier,  there  is  a 
clear  tendency  for  more  experienced  pilots  to  assign 
higher  priorities.  Most  of  the  differences  in  7  have  the 
more  experienced  pilots  assigning  the  higher  priority. 
Novice  pilots  may  have  less  capacity  available  to  deal 
with  all  of  the  important  information,  so  some  of  it 
takes  lower  priority.  It  is  important  to  distinguish 
between  what  should  be  high  priority  and  what  can 
be  high  priority  given  the  pilot’s  ability.  Interestingly 
the  novice  pilots  appear  to  assign  priority  ratings  in 


Phases  of  Flight 


Figure  3.  Priorities,  phase  of  flight,  and  experience. 
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TakeoffExp  —  Hi—  ■  TakeoffNov 


Figure  4.  Takeoff  priorities. 
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Figure  5.  Climb  priorities. 
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Figure  6.  Transition  to  cruise  priorities. 
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Figure  7.  Cruise  priorities. 
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Figure  8.  Descent  priorities. 
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Figure  9.  Approach  priorities. 


Landing  Exp  —  •  Landing  Nov 


17 


Figure  10.  Landing  priorities. 
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Figure  1 1 .  Preflight  planning  priorities. 
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Figure  12.  In-flight  planning  priorities. 


Table  7.  Major  novice-experienced  priority  differences. 


Phase 

Information  Element 

Higher  Priority 
by: 

P 

Takeoff 

Vertical  Velocity 

Nov 

.015 

Climb 

Track 

Exp 

.025 

Traffic/Other  Comm 

Exp 

.043 

Vertical  Velocity 

Nov 

.016 

Transition  to  Cruise 

Ground  Speed 

Nov 

.046 

Cruise 

RPM  (power) 

Exp 

.032 

Time  -  ETA/ETE 

Nov 

.Oil 

Vertical  Velocity 

Nov 

,043 

Descent 

Traffic 

Exp 

.022 

Traffic/Other  Comm 

Exp 

.005 

Course 

Exp 

.028 

Airport  Configuration 

Exp 

.013 

Approach 

General  Weather 

Exp 

.056 

Landing 

Runway  Aim  Point 

Exp 

Traffic/Other  Comm 

Exp 

HSgjll 

ATC  Comm 

Exp 

.043 

Table  8.  Highest  priority  information  by  phase  of  flight. 


Flying 

Planning 

Information  Element 

TO 

CImb 

TTC 

Cruse 

Desc 

App 

Land 

Num 

pp* 

IFP+  Num 

Aircraft  configuration 

X 

X 

X 

X 

4 

X 

1 

Engine  health 

X 

X 

X 

X 

X 

X 

X 

7 

0 

Fuel  quantity 

X 

1 

X 

X 

2 

Fuel  selection 

X 

X 

2 

0 

RPM  (power) 

X 

X 

X 

X 

X 

X 

X 

7 

0 

Altitude  -  AGL 

X 

X 

X 

X 

X 

X 

X 

7 

0 

Altitude  -  MSL 

X 

X 

X 

X 

4 

X 

X 

2 

Distance 

0 

X 

1 

Bank 

X 

X 

X 

3 

0 

Pitch  (attitude) 

X 

X 

X 

X 

X 

5 

0 

Yaw 

X 

X 

X 

3 

0 

Course 

X 

X 

X 

X 

4 

X 

1 

Heading 

X 

X 

X 

X 

4 

X 

1 

Track 

X 

X 

X 

X 
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Waypoints 

X 

1 

X 

X 

2 

Airspeed 

X 

X 

X 

X 

X 

X 

X 

7 

X 

1 

Ground  speed 

X 

1 

X 

1 

Vertical  velocity 

X 

X 

2 

0 

Time  -  ETA/ETE 

0 

X 

X 

2 

Airport  configuration 

X 

X 

X 

3 

X 

X 

2 

Runway  aim  point 

X 

X 

2 

0 

Runway  remaining 

X 

1 

0 

ATC  comm 

X 

X 

X 

X 

X 

X 

X 

7 

X 

1 

Traffic  /  other  comm 

X 

X 

X 

X 

X 

5 

0 

Obstructions 

X 

X 

X 

X 

4 

X 

1 

Traffic 

X 

X 

X 

X 

X 

X 

X 

7 

0 

X 

X 

X 

3 

X 

X 

2 

General  weather 

X 

X 

X 

X 

X 

5 

X 

X 

2 

Wind 

X 

X 

X 

3 

X 

X 

2 

*  Preflight  planning 


+  In-flight  planning 
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line  with  their  capacity  rather  than  with  what  might 
be  the  ideal  priority.  Some  information  elements 
show  experience  differences  in  more  than  one  phase 
of  flight.  Traffic/ Other  Communication  consistently 
receives  higher  priority  ratings  from  the  experienced 
pilots.  In  contrast,  novices  assign  higher  priority 
ratings  to  Vertical  Velocity  in  Takeoff,  Climb,  and 
Cruise.  Perhaps  experienced  flyers  focus  on  factors 
such  as  pitch  and  power  and  let  the  vertical  velocity 
take  care  of  itself,  while  novice  flyers  focus  directly  on 
vertical  velocity.  Instructor  pilots  may  find  it  useful 
to  consider  these  priority  differences  as  they  think 
about  what  topics  to  discuss  with  their  students. 

Table  8  gives  another  view  of  the  priorities  in  the 
ph  ases  of  flight.  The  information  elements  with  a 
mean  priority  rating  of  2  or  better  are  shown  for  each 
phase  of  flight.  The  items  that  are  critical  in  all  of  the 
flying  phases  are  often  available  in  the  central  instru¬ 
ment  cluster,  but  not  all  of  the  critical  information 
elements  are  found  there.  Some  items  have  high 
priority  in  planning,  but  not  in  flying.  It  may  be 
worth  considering  how  to  deal  with  planning  versus 
flying  information  as  displays  are  designed. 

Information  Priorities  and  Workload 

Considerable  attention  has  been  devoted  to  char¬ 
acterizing  and  assessing  workload  both  under  normal 
operating  conditions  and  under  various  unusual  situ¬ 
ations  (Corwin,  1992;  Gopher  &  Donchin,  1986; 
Moray,  Johanssen,  Pew,  Rasmussen,  Sanders,  & 
Wickens,  1979).  Pilots  generally  report  that  workload 
is  greater  in  some  phases  of  flight  than  in  others. 
Takeoff,  approach,  and  landing  are  usually  identified 
as  the  phases  involving  the  highest  workload  under 
normal  operating  conditions.  The  priority  ratings  we 
collected  in  the  present  study  offer  an  additional  way 
to  measure  workload.  We  propose  that  the  average 
priority  rating  for  data  within  a  phase  of  flight  will 
reflect  workload,  namely,  the  higher  the  average 
priority  rating,  the  higher  the  workload.  Another 
measure  from  our  study  is  also  worth  examining  for 
its  relation  to  workload:  the  proportion  of  the  highest 
priority  ratings  in  the  set  of  ratings  for  a  phase  of 
flight.  Presumably,  the  higher  the  proportion  of 
highest-priority  (i.e.,  priority  1)  information  ele¬ 
ments,  the  higher  the  workload. 

To  test  these  ideas,  we  examined  previous  studies 
of  workload  in  the  literature.  One  study,  Corwin 
(1992),  was  particularly  appropriate  for  our  purposes 
because  that  study  assessed  workload  both  in-flight 


and  post-flight  for  three  simulator  flights.  One  flight 
was  a  30-minute  flight  between  two  airports.  A  sec¬ 
ond  30-minute  flight  was  similar,  with  the  addition 
of  communications.  The  third  1.5-hour  flight  in¬ 
volved  malfunctions  with  two  diversions,  an  engine 
failure,  and  a  hydraulic  system  failure.  Using  the 
Subjective  Workload  Assessment  Technique  (SWAT), 
workload  was  assessed  both  in-flight  and  post-flight 
for  each  of  several  phases  of  flight  (takeoff,  climb,  top 
of  climb,  cruise,  top  of  descent,  approach,  and  land¬ 
ing).  These  phases  match  up  very  well  with  the  flying 
phases  we  defined  for  our  priority  ratings  (takeoff, 
climb,  transition  to  cruise,  cruise,  descent,  approach, 
and  landing).  The  values  of  the  various  measures  are 
shown  in  Table  9,  along  with  the  correlations  be¬ 
tween  these  measures.  Interestingly,  the  measures 
derived  from  the  priority  ratings  of  the  present  study 
correlate  about  as  well  with  the  measures  from  the 
Corwin  study  as  those  measures  correlate  with  each 
other.  SWAT  measures  were  more  highly  correlated 
with  the  percentages  of  priority  ratings  falling  in  the 
top  priority  category  than  they  were  with  the  mean 
priority  ratings.  This  result  suggests  that  workload 
may  derive  primarily  from  high-priority  data  require¬ 
ments,  rather  than  from  the  total  potential  data 
requirements. 

While  this  is  a  rather  appealing  notion,  it  should 
be  kept  in  mind  that  this  is  based  upon  correlational 
data,  and  thus  no  causality  can  be  directly  inferred 
from  these  observations.  A  slightly  stronger  case  can 
be  made,  however,  if  one  looks  at  additional  sources 
of  data,  i.e.,  the  data  requirements  per  flight  phase 
and  the  distribution  of  accidents  by  flight  phase. 
Accidents  are  distributed  disproportionately  across 
ph  ases  of  flight  such  that  the  frequency  of  incidents 
and  accidents  on  take-off  and  approach/landing  is  far 
greater,  by  a  significant  amount,  than  the  times  of 
exposure  to  these  phases  of  flight.  Although  several 
factors  are  thought  to  influence  this  distribution,  it  is 
widely  accepted  that  workload  is  higher  during  take¬ 
off  or  approach  than  it  is  during  level  cruise.  Exami¬ 
nations  of  information  requirements  by  phase  of 
flight  indicate  that  changes  in  aircraft  configuration, 
increases  in  communications  requirements,  increases 
in  traffic  in  the  terminal  area,  and  requirements  for 
maintaining  separation  all  contribute  to  an  elevated 
workload  during  the  approach  and  landing.  It  is 
documented  that  more  data  are  needed  per  unit  time 
during  an  approach  and  landing  than  are  accessed 
during  a  cruise  flight  segment.  If  we  can  assume  that 
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Table  9.  Workload  measures  and  correlations. 


A 

B 

C 

Exper, 

Mean 

Exper. 

Percent 

Corwin 

Basic 

Phase 

Priority 

Rating 

Top 

Priority 

In- 

Flight 

Takeoff 

2.06 

53.7% 

24.9 

Climb 

2.15 

41.6% 

11.1 

Trans’n  to  Cruise 

2.50 

24.4% 

9.1 

Cruise 

2.30 

28.9% 

2.9 

Descent 

2.10 

36.4% 

16.8 

Approach 

1.74 

59.1% 

15.9 

Landing 

1.83 

62.9% 

30.4 

Correlations 

A 

B 

C 

A 

-0.94 

-0.67 

B 

-0.94 

0.82 

C 

-0.67 

0.82 

D 

-0.71 

0.84 

0.96 

E 

-0.72 

0.90 

0.93 

F 

-0.66 

0.83 

0.95 

G 

-0.89 

0.91 

0.73 

H 

-0.77 

0.77 

0.65 

higher-priority  data  are  “required”  whereas  lower- 
priority  data  are  optional  in  some  cases,  then  we  can 

forge  a  link  between  the  percentage  of  information 
elements  rated  as  high-priority  and  the  expected 
workload  resulting  from  the  increased  information¬ 
processing  requirements. 

Project  3:  The  Mental  Organization  of 
Information  Accessed  in  Flight 

In  this  project,  we  used  network  scaling-methods 
to  analyze  how  pilots  conceptually  organize  the  infor¬ 
mation  used  in  flight.  First,  we  present  some  back¬ 
ground  on  network  scaling. 

Network  Analyses  of  Relatedness  Data 
The  techniques  used  in  the  present  analysis  are 
generally  known  as  Pathfinder  Network  Scaling  (see 
Schvaneveldt,  1990).  The  method  uses  individuals’ 
judgments  as  a  source  from  which  to  extract  underly¬ 
ing  network  structures.  This  method  has  been  used  to 
capture  expert-novice  differences  in  conceptual  struc¬ 
ture  (USAF  pilots  -  Schvaneveldt,  et  al.,  1985;  also 
see  Cooke  &  Schvaneveldt,  1988  and  Rowe,  Cooke, 
Hall  &  Halgren,  1996),  to  assess  student  knowledge 


D 

E 

F 

G 

H 

Corwin 

Corwin 

Corwin 

Corwin 

Corwin 

Basic 

Conun 

Comm 

Malfn 

Malfn 

Post- 

In- 

Post- 

In- 

Post- 

Flight 

Flight 

Flight 

Flight 

Flight 

25.6 

24.5 

27.8 

20.0 

16.3 

14.4 

19.6 

15.6 

14.7 

12.7 

3.9 

11.7 

10.2 

1.4 

2.9 

1.3 

8.6 

11.4 

14.1 

21.3 

16.9 

15.2 

16.8 

13.1 

18.2 

14.9 

19.3 

17.1 

32.2 

33.7 

26.5 

29.4 

29.0 

46.4 

59.9 

D 

E 

F 

G 

H 

-0.71 

-0.72 

-0.66 

-0.89 

-0.77 

0.84 

0.90 

0.83 

0.91 

0.77 

0.96 

0.93 

0.95 

0.73 

0.65 

0.94 

0.95 

0.68 

0.56 

0.94 

0.93 

0.78 

0.66 

0.95 

0.93 

0.74 

0.64 

0.68 

0.78 

0.74 

0.96 

0.56 

0.66 

0.64 

0.96 

(Goldsmith,  Johnson,  &  Acton,  1991),  and  to  ana¬ 
lyze  and  design  user-system  interfaces  (McDonald  & 
Schvaneveldt,  1988). 

Graph  Theory  and  Relations 
Among  Information  Elements 
The  abstract  model  underlying  the  proposed  model 
of  relationships  among  information  elements  is  graph 
and  network  structures.  In  mathematical  graph  theory, 
a  graph  is  an  abstraction  consisting  of  a  set  of  nodes 
and  a  set  of  pairs  of  the  nodes  (Harary,  1969).  Each 
such  pair  of  nodes  is  called  a  link.  The  links  can  be 
directed  or  undirected.  The  nodes  represent  some 
entity  (concept,  document,  individual,  process,  etc.) 
and  the  links  represent  relationships  between  nodes. 
A  network  is  a  graph  with  weights  (or  costs)  associ¬ 
ated  with  the  links.  We  often  expand  these  basic 
definitions  to  include  distinguishing  different  types 
of  nodes  and  different  types  of  links.  The  present 
application  can  benefit  from  the  ability  to  systemati¬ 
cally  distinguish  different  node  and  link  types.  For 
example,  node  types  can  be  used  to  represent  collec¬ 
tions  of  related  nodes,  which,  for  some  purposes,  may 
be  treated  as  a  single  category  (e.g.,  the  various 
parameters  relating  to  the  status  of  engines). 
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METHOD 

Pathfinder  scaling  requires  judgments  of  related¬ 
ness  for  the  pairs  of  elements  to  be  scaled.  Pathfinder 
produces  a  network  showing  the  connections  be¬ 
tween  the  various  elements.  The  network  structures 
also  reveal  clusters  of  elements  that  are  interrelated. 
Because  the  relatedness  judgments  require  pilots  to 
evaluate  all  pairs  of  the  elements,  the  number  of 
elements  must  be  limited.  We  selected  the  informa¬ 
tion  elements  shown  in  Table  10  for  study  in  this 
project.  To  limit  the  number  of  items,  we  excluded 
information  more  related  to  airports,  while  retaining  the 
items  related  to  flying  the  aircraft  and  to  communications. 

Participants 

Thirty-four  certified  pilots  of  varying  age  and 
experience  were  used  to  obtain  the  relatedness  rating 
data.  A  number  of  subjects  overlap  from  the  priority¬ 
rating  task  to  the  relatedness-rating  task.  The  partici¬ 
pants  were  obtained  from  diflferent  areas  around  the 
United  States.  To  examine  the  impact  of  experience 
on  judgments  of  relatedness,  we  separated  the  pilots 
into  two  groups  based  on  their  total  flying  time.  The 
hours  of  the  1 5  less  experienced  pilots  ranged  from 
100  to  950  with  a  mean  of  372  and  a  standard 
deviation  of 323.  The  19  more  experienced  pilots  had 
a  range  of  flying  time  from  1,000  to  20,400  hours 
with  a  mean  of  5)320  and  a  standard  deviation  of 
5,143.  Thirteen  percent  of  the  less-experienced  pilots 
held  the  Instrument  Rating  while  74%  of  the  more- 
experienced  pilots  were  instrument  rated. 

Materials 

The  relatedness-rating  task  was  conducted  using 
two  Hewlett-Packard  OmniBook  600c  laptop  com¬ 
puters.  A  MS/DOS  program  presented  instructions 
and  all  of  the  231  pairs  of  the  22  information  ele¬ 
ments  for  relatedness  ratings. 

Design  and  Procedure 

Collection  of  the  relatedness  data  occurred  over  an 
eight-month  period.  The  participants  completed  a 
demographic  questionnaire  (see  Appendix  1).  After 
the  demographic  form  was  completed,  the  partici¬ 
pants  were  directed  to  the  computer  for  the  related¬ 
ness  ratings.  After  reviewing  the  22  information 
elements  to  be  rated,  the  pilots  rated  the  relatedness 
for  each  of  the  pairs  of  elements.  The  23 1  pairs  were 


Table  10.  The  22  information  elements  used  for 
relatedness  ratings. 

Aircraft  Configuration 

Airspeed 

Engine  Health 

Ground  Speed 

Fuel  Status 

Vertical  Velocity 

RPM  (power) 

Time  (ETA/ETE) 

Altitude 

ATC  Comm 

Distance  to  Waypoint 

Traffic  Comm 

Bank 

Obstructions 

Pitch 

Traffic 

Yaw 

Airspace 

Course 

General  Weather 

Track 

Wind 

presented  in  a  different  random  order  for  each  par¬ 
ticipant.  At  the  conclusion  of  the  rating  task,  the 
project  was  discussed  with  the  pilots,  and  they  were 
encouraged  to  provide  feedback  about  the  rating  task 
and  the  objectives  of  the  study. 

RESULTS  AND  DISCUSSION 

The  Pathfinder  network  derived  from  the  average 
of  all  of  the  relatedness  rating  data  is  shown  in  Figure 
13.  The  figure  generally  reveals  which  items  of  infor¬ 
mation  are  most  closely  related  to  which  other  items 
by  virtue  of  the  links  in  the  figure.  Linked  nodes 
indicate  a  close  relationship  between  the  items  linked, 
judged  by  the  number  of  links  connected  to  the 
nodes.  Course,  Altitude,  Airspeed,  and  Power  are  the 
central  nodes  in  the  figure.  Arguably,  these  items  are 
also  central  to  the  flight  task.  Airspeed  is  required  for 
lift.  Power  is  essential  to  produce  Airspeed.  Course 
and  Altitude  are  essential  parts  of  navigation.  The 
other  information  elements  all  tie  to  these  core  ele¬ 
ments  to  give  a  more  complete  picture  of  the  informa¬ 
tion  required  in  flight.  Thus,  the  picture  of  the 
organization  of  the  information  revealed  by  the  net¬ 
work  analysis  generally  corresponds  with  intuitive 
impressions,  as  it  is  intended  to  do. 

Table  1 1  gives  more  detail  about  the  close  connec¬ 
tions  among  information  elements.  The  table  shows 
the  most  closely  related  pairs  of  items  based  on 
judged  relatedness  (rel)  and/or  the  frequency  with 
which  the  pairs  are  linked  in  the  networks  of  34 
individual  pilots  (net). 
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Figure  13.  The  pathfinder  network  for  average  data  (q=2,  r=8). 
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Table  1 1 .  Most  highly  connected  pairs  of  information  elements. 


Net  Rel 

Pair 

Net 

Rel 

Pair 

32 

8.8 

Time  (ETA/ETE) 

Ground  Speed 

16 

6.5 

Wind 

Distance  to  Waypoint 

31 

8.6 

Wind 

Ground  Speed 

11 

7.0 

General  Weather 

Altitude 

30 

8.6 

Time  (ETA/ETE) 

Distance  to  Waypoint 

13 

7.0 

Track 

Distance  to  Waypoint 

30 

8.4 

Obstructions 

Altitude 

11 

7.0 

Ground  Speed 

Track 

29 

8.4 

RPM  (power) 

Engine  Health 

15 

7.0 

Vertical  Velocity 

Aircraft  Configuration 

28 

8.4 

Track 

Course 

11 

6.9 

Wmd 

Altitude 

27 

8.3 

Traffic 

Traffic  Comm 

13 

6.9 

Ground  Speed 

Course 

27 

8.4 

Airspace 

ATC  Comm 

13 

6.9 

ATC  Comm 

Altitude 

27 

8.1 

Wind 

Track 

13 

6.8 

Ground  Speed 

Aircraft  Configuration 

26 

8.2 

Traffic  Comm 

ATC  Comm 

9 

6.8 

General  Weather 

Course 

25 

8.3 

Airspeed 

RPM  (power) 

15 

6.3 

Pitch 

Bank 

25 

8.3 

Time  (ETA/ETE) 

Airspeed 

15 

5.5 

Yaw 

Bank 

25 

8.3 

Wind 

Time  (ETA/ETE) 

14 

6.8 

Obstructions 

Vertical  Velocity 

25 

8.2 

Wmd 

General  Weather 

13 

6.7 

Traffic 

Altitude 

24 

7.9 

Airspeed 

Pitch 

10 

6.7 

Course 

Fuel  Status 

24 

8.2 

Vertical  Velocity 

Pitch 

7 

6.7 

Airspeed 

Altitude 

24 

8.0 

Time  (ETA/ETE) 

Fuel  Status 

14 

6.6 

Pitch 

Aircraft  Configuration 

24 

7.8 

Airspace 

Traffic 

13 

6.6 

Airspeed 

Engine  Health 

23 

7.9 

Airspeed 

Aircraft  Configuration 

11 

6.6 

Airspeed 

Fuel  Status 

23 

8.4 

Ground  Speed 

Airspeed 

12 

6.6 

Pitch 

RPM  (power) 

22 

7.8 

Airspace 

Altitude 

7 

6.6 

Traffic  ' 

Course 

21 

7.5  Distance  to  Waypoint 

Fuel  Status 

10 

6.6 

General  Weather 

Time  (ETA/ETE) 

21 

7.6 

Time  (ETA/ETE) 

Track 

9 

6.5 

General  Weather 

Airspace 

20 

8.0 

Airspace 

Traffic  Comm 

13 

6.4 

Time  (ETA/ETE) 

Engine  Health 

20 

7.7 

Traffic 

ATC  Comm 

12 

6.3 

Fuel  Status 

Engine  Health 

20 

7.7 

Wind 

Course 

10 

6.3 

Ground  Speed 

Altitude 

20 

6.7 

Wmd 

Airspeed 

11 

6.3 

Ground  Speed 

Pitch 

19 

7.9 

Wmd 

Fuel  Status 

12 

6.3 

ATC  Comm 

Course 

19 

7.9 

Ground  Speed 

RPM  (power) 

13 

6.3 

RPM  (power) 

Aircraft  Configuration 

19 

7.8 

Obstructions 

Course 

9 

6.3 

Course 

Altitude 

17 

7.7 

Course 

Distance  to  Waypoint 

10 

6.3 

Obstructions 

Pitch 

19 

7.7 

Altitude 

RPM  (power) 

8 

6.2 

Traffic  Comm 

Altitude 

19 

7.6 

Ground  Speed 

Distance  to  Waypoint 

10 

6.2 

Airspace 

Obstructions 

18 

7.6 

Airspeed 

Distance  to  Waypoint 

6 

6.2 

General  Weather 

Ground  Speed 

18 

7.6 

Vertical  Velocity 

RPM  (power) 

12 

6.1 

Altitude 

Engine  Health 

17 

7.4 

Vertical  Velocity 

Altitude 

10 

6.1 

Airspeed 

Bank 

17 

7.3 

Time  (ETA/ETE) 

RPM  (power) 

10 

6.1 

Airspace 

Airspeed 

17 

7.3 

Obstructions 

Track 

6 

6.1 

Altitude 

Aircraft  Configuration 

17 

7.3 

Pitch 

Altitude 

10 

6.  IDistance  to  Waypoint 

RPM  (power) 

16 

7.2 

Time  (ETA/ETE) 

Course 

8 

6.1 

General  Weather 

Obstructions 

14 

7.2 

Airspace 

Course 

9 

6.0 

Airspace 

Track 

12 

7.2 

RPM  (power) 

Fuel  Status 

9 

5.9 

Ground  Speed 

Fuel  Status 

12 

7.2 

Vertical  Velocity 

Airspeed 

11 

5.9 

ATC  Comm 

Time  (ETA/ETE) 

13 

7.2 

General  Weather 

Fuel  Status 

9 

5.9 

Airspace 

Ground  Speed 
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Clusters  of  Information  Elements 

The  results  of  a  hierarchical  cluster  analysis  are 
shown  in  Figure  14,  Bold  lines  surrounding  the 
information  elements  show  clusters.  The  cluster  analy¬ 
sis  reveals  larger  groupings  of  the  information 
elements,  which  may  be  useful  in  suggesting  organi¬ 
zations  of  information  displays.  The  clusters  gener¬ 
ally  identify  meaningful  hierarchical  groupings  of  the 
information  elements. 

The  information  clusters  suggest  how  pilots  reach 
decisions  by  considering  multiple  factors.  For  ex¬ 
ample,  the  clustering  of  Distance  to  Waypoint  and 
Fuel  Status  illustrates  the  decision-making  about  the 
likelihood  of  reaching  a  waypoint.  This  cluster  is 
closely  connected  to  the  cluster  including  Time  and 
Ground  Speed  and  Wind,  reflecting  the  relevance  of 
all  of  these  factors  in  decision-making  in  flight. 
Presenting  such  information  in  an  arrangement  and 
a  format  that  facilitates  decision-making  should  be  of 
considerable  value  to  pilots. 


Experience  and  Networks 

We  examined  the  differences  between  the  network 
derived  from  the  ratings  of  novice  pilots  and  that 
derived  from  the  ratings  of  experienced  pilots.  The 
networks  were  generated  with  r  =  ¥  and  q  =  2  which 
yields  networks  with  the  maximum  number  of  links 
given  ordinal  data  (Schvaneveldt,  Durso,  &  Dearholt, 
1989).  The  experienced-pilots’  network  had  28  links 
while  the  novices’  network  had  29.  In  general,  the 
similarities  in  the  two  networks  were  more  striking 
than  the  differences.  The  two  networks  had  21  links 
in  common.  The  differences  between  the  networks 
are  shown  in  Figure  15.  Many  were  simply  differ¬ 
ences  in  the  linking  among  three  items.  Such  differ¬ 
ences  in  triples  of  items  are  not  particularly  significant 
because  the  three  items  are  joined  in  both  networks 
albeit  with  different  links.  It  is  possible,  however, 
that  the  different  linking  points  to  subtle  differences 
in  the  way  novices  and  experienced  pilots  think  about 
the  information  involved.  In  Figure  15,  the  triangles 
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Figure  15-  Differences  in  novice  and  experienced  pilots*  networks. 


shown  include  the  links  shared  by  both  networks  to 
highlight  the  triangles.  The  other  link  differences 
cannot  be  accounted  for  by  such  triples  suggesting 
greater  differences  in  the  perception  of  relatedness 
between  the  two  groups  of  pilots. 

CONCLUSIONS,  APPLICATIONS, 
RECOMMENDATIONS,  AND  CAVEATS 

The  data  presented  in  this  report  provide  a  picture 
of  how  information  functions  in  the  National  Air¬ 
space  System,  including  how  the  information  is  pri¬ 
oritized  and  organized  by  pilots.  The  views  of  information 
priorities  show  how  priorities  change  with  phases  of 
flight  and  how  pilot  experience  relates  to  the  perception 
of  priorities.  The  views  of  information  organization 


reveal  some  of  the  ways  pilots  inter-relate  the  various 
elements  of  information.  It  is  difficult  to  determine 
the  extent  to  which  the  experience  differences  in  this 
study  are  due  to  experience  per  se  as  opposed  to 
selection  that  occurs  over  the  flying  careers  of  pilots. 
We  know  that  pilot  attrition  may  be  due  to  a  number 
of  factors,  ranging  from  choosing  to  stop  flying  to 
having  a  fatal  accident.  Is  it  that  the  pilots  who 
endure  have  different  perceptions  of  priorities  to 
begin  with,  or  do  pilots  learn  to  develop  these  priori¬ 
ties  as  a  result  of  experience?  It  may  well  be  that  both 
of  these  factors  operate.  A  longitudinal  study  would 
provide  some  answers  to  these  questions.  Regardless 
of  how  the  experience  issue  is  resolved,  the  data 
reported  here  should  be  of  value  in  designing  infor¬ 
mation  display  systems. 
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This  study  has  focused  on  information  priorities, 
information  organization,  and  information  sources. 
There  are  certainly  several  other  factors  that  are 
important  in  the  design  of  information  displays, 
including  issues  of  perception,  format,  analog  vs. 
digital,  etc.  As  new  display  solutions  are  sought,  it  is 
important  to  keep  in  mind  the  value  of  redundancy. 
Limited  panel  real  estate  places  a  severe  constraint  on 
just  how  much  can  be  placed  before  the  pilot,  but 
some  solution  to  the  problem  of  providing  alterna¬ 
tive  sources  of  information  must  be  found  in  the 
interest  of  safety.  We  have  not  emphasized  this  issue 
in  the  report,  but  our  analysis  of  sources  using  the 
instruments  found  in  most  general  aviation  aircraft 
shows  the  redundancy  existing  in  the  present  system. 
New  systems  must  strive  to  maintain  redundancy  to 
ensure  pilot  confidence. 

There  are  numerous  ways  that  data  on  pilot  in¬ 
formation  organization  and  information  priorities 
could  be  used  to  enhance  the  safety  of  operations  in 
the  national  airspace.  These  data  can  potentially  be 
used  to  position  instrumentation  in  the  cockpit, 
design  integrated  cockpit  information  systems  (ICIS), 
restructure  the  logic  of  and  displays  for  flight  man¬ 
agement  systems  (FMS),  modify  communications 
between  pilots  and  ATC/ATM  (both  voice  and 
datalink),  reduce  workload,  and  even  to  refine  meth¬ 
ods  and  curricula  for  flight  instruction. 

Instrumentation  Layout  and  Integrated 
Cockpit  Information  Systems 

Given  that  one  has  a  good  idea  of  how  information 
elements  are  prioritized  and  related  in  the  perception 
of  the  pilot,  it  should  be  helpful  to  use  these  findings 
to  determine  how  data  are  presented  in  the  cockpit  in 
a  way  that  would  enhance  processing  of  those  data, 
both  in  terms  of  speed  and  accuracy.  For  example, 
one  can  see  from  the  hierarchical  clusters  shown  in 
Figure  14  that  fuel  status  and  distance  to  waypoint 
are  clustered,  and  are  closely  tied  to  ETA/ETE  and 
ground  speed.  This  might  lead  to  recommendations 
that  displays  for  related  parameters  be  closely  grouped 
or,  better  yet,  integrated  for  more  effective  presenta¬ 
tion.  Thus,  one  could  argue  the  case  that  one  way  to 
facilitate  data  assimilation  would  be  to  present  a 
graphic  showing  aircraft  location  on  a  plan-view 
map,  with  range  rings  or  vectors  indicating  waypoints/ 
destination  and  instantaneous  “fuel”  range,  both 
absolute  and  with  45-minute  reserve.  The  clustered 


data  are  now  effectively  presented  in  an  integrated 
display  that  allows  direct  and  rapid  interpretation  of 
the  situation. 

Flight-Management  Systems 

Several  studies  (i.e.,  Funk  et  al.,  1999;  Degani, 
Shafto  and  Kirlik,  1999)  have  found  that  pilots 
perceive  flight-management  systems  as  problematic 
in  a  number  of  ways.  Two  contributing  factors  to  this 
attitude  have  been  identified:  (1)  the  organization 
and  grouping,  or  lack  thereof,  of  functions  and  data; 
and  (2)  the  failure  of  the  system  to  adequately  com¬ 
municate  intent  to  the  pilot.  The  first  is  probably  the 
most  amenable  to  modification  using  clustering/ 
connections  data,  and  some  efforts  are  presently 
underway  (Riley,  DeMers,  &  Misiak,  1998)  to  re¬ 
structure  the  data-entry  interface  more  along  the 
lines  of  natural  language,  aping  the  way  in  which 
instrument  clearances  are  verbally  delivered  and  read 
back.  This,  to  some  degree,  involves  a  reorganization 
of  the  data  in  ways  that  are  more  consistent  with  pilot 
mental  models  and  constructs.  The  present  data  can 
be  used  in  such  endeavors  to  group  data  for  input  to 
and  retrieval  from  the  FMS  in  a  more  natural  and 
expected  fashion,  which  should  reduce  entry  times 
and  errors. 

The  second  problem  of  communicating  intent 
may  also  involve  restructuring  reports  of  FMS  activ¬ 
ity  in  such  a  way  as  to  be  more  easily  associated,  by  the 
pilot,  with  the  greater  context  of  the  flight.  Although 
there  is  a  significant  argument  that  the  best  FMS 
should  be  one  in  which  there  are  no  modes  but, 
rather,  an  apparently  seamless  operation  from  one 
flight  regime  to  the  next,  the  FMS  intent  still  must  be 
communicated  in  the  context  of  current  flight  condi¬ 
tions  (i.e.,  messaging  priorities  are  likely  to  change 
from  takeoff  through  cruise  to  landing,  and  the 
system  should  organize  and  present  action-related 
data  in  the  appropriate  context  and  with  the  appro¬ 
priate  priority  weighting). 

Controller-Pilot  Communications  and 
Datalink 

Our  argument  is  that  the  organization  of  data  is 
important  across  the  board  to  achieve  efficient,  timely, 
and  accurate  responses  by  both  the  airborne  and 
ground-based  systems.  Thus,  the  same  groupings  and 
organization  applied  to  flight-control  data  can  also 
be  applied  to  many  other  forms  of  flight-related  data 
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(e.g.,  Riley  et  al.,  1998).  Domain-specific  analyses 
will  be  necessary,  however,  to  verify  that  each  of  the 
users  is  receiving  data  in  formats  appropriate  to  their 
task.  This  should  apply  to  data  regarding  weather, 
airports,  traffic,  obstacles,  and  clearances,  to  name  a  few. 

Flight  Instruction  Curricula 

One  question  that  has  not  been  addressed  directly 
here  is  the  issue  of  how  data  should  be  organized  in 
the  most  abstract  sense  versus  how  data  are  organized. 
It  is  clear  that  the  actual  task  structure  (how  we  fly 
aircraft  and  how  the  data  are  presented  as  a  function 
of  aircraft  certification  requirements)  largely  deter¬ 
mines  how  instruction  is  structured. 

However,  it  is  not  at  all  clear  that  the  way  in  which 
data  are  presently  organized  and  presented  is  neces¬ 
sarily  optimal.  The  question,  which  could  be  pursued 
here,  is  what  organization  may  be  imposed  upon 
fundamental  information  regarding  aircraft  flight 
(rate  of  travel,  altitude,  heading,  position  relative  to 
desired  location  in  space  or  final  goal)  by  nonpilots  as 
a  function  of  their  concept  of  the  natural  environ¬ 
ment  and  the  inherent  implied  organization  via  physi¬ 
cal  features  and  properties.  Cases  have  already  been 
documented  (Beringer,  1978)  in  which  pilot  behav¬ 
iors  were  being  established  during  training  that  were 
counter  to  nonpilot  stereotype  responses,  leaving  the 
door  open  for  reversion  to  those  now  inappropriate 
behaviors  under  stress. 

The  same  can  be  asked  of  the  organization  of  data 
for  the  flight  task:  To  what  degree  may  we  be  distort¬ 
ing  any  inherent  organizational  structures  that  exist 
in  the  general  population  in  subsequent  flight  train¬ 
ing,  and  how  might  task  and  data  structures  be  modified 
to  take  better  advantage  of  extant  knowledge? 

Regardless  of  how  these  issues  are  resolved,  flight 
instructors  may  benefit  from  knowing  how  more  and 
less  experienced  pilots  differ  in  information  priorities 
and  in  the  mental  organization  of  that  information. 

An  Example:  Applying  the  Data  to 
Display  Allocation/Design 
One  specific  example  of  how  these  data  might  be 
applied  can  be  found  in  the  design  of  display  suites 
for  the  new  GA  glass  cockpit,  which  are  expected  to 
include  a  Primary  Flight  Display  PFD)  and  a  Multi¬ 
function  Display  (MFD).  The  data  demonstrate  that 
pilots  require  different  information  sets  depending 


on  the  mode  of  flight.  Thus,  it  is  expected  that 
although  common  information  may  be  present  across 
two  or  more  modes,  the  relative  importance  of  the 
information  will  change.  Consequently,  a  display- 
decluttering  scheme  should  take  into  account  the 
most  salient  pieces  of  data  and  preserve  those  over 
other  less  task-relevant  data. 

Assume,  for  example,  that  we  are  allocating  display 
space  in  such  a  way  that  we  can  depict  only  the  six 
highest-priority  items  (the  assumption  of  display 
limitations  is  real;  the  choice  of  6  is  artificial).  Con¬ 
centrating  on  takeoff  and  drawing  from  the  experi¬ 
enced-pilots’  rankings  presented  in  Figure  4,  we 
would  select  RPM/power,  wind,  engine  health,  air¬ 
craft  configuration,  airspeed,  and  general  weather  for 
presentation.  Next,  we  can  examine  Figure  14  to 
determine  those  elements  that  naturally  cluster  to¬ 
gether  and  may  be  suited  for  integrated  presentation. 
We  find  that  airspeed  and  aircraft  configuration 
cluster  at  a  high  level,  with  RPM/power  and  engine 
health  forming  another  high-level  cluster,  with  these 
two  clusters  in  close  proximity  and  with  a  direct  link. 
Wind  and  general  weather  can  be  seen  as  another 
cluster  with  a  link  to  the  first  (the  one  containing 
airspeed).  This  suggests  that  there  could  be  three 
display  subgroups  derived  from  these  data,  each  con¬ 
sisting  of  two  data  variables.  The  degree  to  which  the 
individual  pieces  can  be  woven  into  an  integrated 
presentation  will,  of  course,  affect  formatting,  and  it 
should  be  kept  in  mind  that  the  data  in  this  report  do 
not  specifically  designate  a  particular  format.  Rather, 
the  data  reveal  the  priority  of  information  elements 
from  the  pilot’s  perspective. 

One  can  then  apply  some  human  factors  engineer¬ 
ing  in  this  example,  however,  and  see  that  an  inte¬ 
grated  presentation  of  applied  power  or  thrust  that 
includes  some  status  indicator  depicting  engine  health 
would  be  a  good  start.  [Example:  A  bargraph  presen¬ 
tation  showing  percent  thrust  with  color  of  the 
bargraph  or  texture  fill  indicating  the  status  of  the 
power  plant.]  Thus,  one  component  of  the  display  is 
defined.  Second,  a  component  presenting  airspeed 
and  aircraft  configuration  (airspeed  is  an  index  of 
when  gear  and  flaps  can  be  extended/ retracted)  would 
be  a  next  step,  with  direct  graphical  indices  or  icons 
representing  the  status  of  gear/flaps/spoilers  and  some¬ 
how  directly  associated  with  the  airspeed  scale  or  indi¬ 
cation.  Finally,  one  could  have  a  third  representation 
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depicting  surface  wind  graphically  related  to  direc¬ 
tion  of  travel  (expected  to  be  runway  heading  on 
takeoff).  These  elements  should  therefore  satisfy  the 
requirements  given  the  restrictions  assumed. 

One  may  have  the  opportunity,  however,  to  ap¬ 
proach  the  display  problem  from  the  other  direction 
given  effective  integration  techniques  and  boundless 
display  area.  This  approach  would  focus  on  provid¬ 
ing  all  of  the  information  elements  ranked  in  the  first 
two  categories  (1  =  critical  and/or  frequently  ac¬ 
cessed;  2  =  important  and/or  usually  accessed).  Go¬ 
ing  back  to  our  previous  example,  one  might  then 
choose  to  include  any  data  that  achieved  a  ranking 
value  of  2.5  or  smaller  (everything  from  Ts  all  of  the 
time  to  2’s  half  of  the  time  and  3’s  half  of  the  time). 
This  approach  would  then  cause  the  first  22  items  (as 
rated  by  experienced  pilots)  in  Figure  4  to  be  in¬ 
cluded  for  display-system  presentation  (all  the  way 
down  to  “airport  configuration”).  This  approach 
would  be  more  suited  to  selecting  data  for  the  entire 
display  suite,  as  opposed  to  selecting  items  for  a  small 
subset  of  the  display  system.  Grouping  would  be 
pursued  as  previously  by  looking  at  clusters  and  links. 
Again,  it  should  be  kept  in  mind  that  the  exact  form 
of  the  data  displays  will  be  governed  by  established 
human  factors  display-formatting  principles,  guide¬ 
lines,  and  ultimately  the  standards  presently  in  force 
for  the  specific  systems  hosting  the  displays. 

Prospectus 

In  this  research,  the  emphasis  has  been  on  identi¬ 
fying  information  priorities  and  information  organi¬ 
zation.  It  would  be  valuable  to  take  the  work  reported 
here  as  a  starting  point  for  an  examination  of  the  kind 
of  decisions  pilots  make  in  the  course  of  flight.  In 
particular,  the  decisions  could  be  examined  from  the 
perspective  of  the  information  that  goes  into  the 
decisions.  Our  analysis  of  clusters  of  information 
elements  points  in  that  direction,  but  it  would  be 
informative  to  pursue  the  involvement  of  informa¬ 
tion  in  decisions  in  greater  depth. 

Finally,  this  research  effort  has  focused  on  normal 
flight  operations  in  contrast  to  unusual  situations  or 
emergencies.  Clearly,  the  work  is  incomplete  without 
including  these  situations.  We  are  in  the  process  of 
remedying  this  lack  in  a  second  investigation  that  will 
focus  on  a  related  analysis  of  information  priorities  in 
emergencies. 
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APPENDIX  A 
Demographic  Questionnaire 


Aviation  Research  Group 
New  Mexico  State  University 

1.  Please  estimate  your  logged  hours  in  each  of  the  following  categories  of  aircraft: 

Single  Engine  Land _ 

Multi  Engine  Land  _ 

Other  _  Types  _ 

2.  Please  list  the  year  in  which  you  obtained  the  following  ratings  or  certificates: 

Private  Pilot  _ 

Instrument  _ 

Commercial  _ 

CFI  _ 

CFII  _ 

ATP  _ 

3.  Approximately  how  many  hours  have  you  logged  in  the  last  12  months? _ 

4.  How,  where,  and  when  did  you  first  learn  to  fly? 

Year  Location 

_  Individual  Flight  Instructor  _  _ 

_  Civilian  Flight  School  _  _ 

_  Military  Flight  School  _  _ 

_  Other  _  _ 
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APPENDIX  B 

Instructions  for  Priority  Ratings  and  the  Priority  Rating  Form 


Aviation  Research  Group 
New  Mexico  State  University 
Las  Cruces,  NM 
Roger  Schvaneveldt,  Director 
The  Priorities  and  Organization  of  Information 
Accessed  by  Pilots  in  Various  Phases  of  Flight 


Task  Description 

We  are  developing  an  analysis  of  the  priorities  associated  with  various  kinds  of  information  in  various  phases  of 
flight.  At  present,  we  are  focusing  on  the  priorities  for  normal  operations  as  opposed  to  emergencies.  You  will 
greatly  assist  us  in  this  effort  by  providing  us  with  your  judgments  of  information  priorities. 

On  the  next  page,  you  will  find  some  questions  about  your  experience  in  aviation.  These  data  will  help  us 
compare  the  priorities  of  pilots  with  varying  experience.  On  the  page  following  is  a  list  of  the  information 
elements  along  with  our  definitions  of  these  elements.  Study  the  list  so  you  will  understand  what  we  are  trying  to 
get  at  with  your  priority  ratings. 

Following  the  list  is  a  sheet  with  the  information  elements  on  the  rows  and  phases  of  flight  on  the  columns.  You 
are  to  enter  your  priority  ratings  in  the  cells.  Most  pilots  find  it  best  to  work  down  each  column  rating  the 
importance  of  the  element  for  that  phase  of  flight.  We  have  also  included  two  planning  phases:  Preflight 
Planning  and  In-flight  Planning.  Because  you  can  give  priorities  associated  with  planning  in  these  two  columns, 
please  rate  the  importance  of  the  information  elements  in  the  flying  phases  relative  to  flying  as  opposed  to 
planning.  The  flying  phases  include:  Takeoff,  Climb,  Transition  to  Cruise,  Cruise,  Descent,  Approach,  and 
Landing.  Try  to  assign  priority  ratings  that  cover  both  IFR  and  VFR  flights.  That  is,  give  the  highest  priority 
for  the  information  element  considering  both  IFR  and  VFR  flights. 

Use  priority  ratings  as  follows: 


Priority 

1 

2 

3 

blank 


Description 

Critical  and/or  Frequently  Access 
Important  and/or  Usually  Accessed 
Relevant  and/or  Sometimes  Accessed 
Not  Relevant  or  Rarely  Accessed 


If  you  have  comments  or  questions,  you  can  reach  Dr.  Schvaneveldt  via  e-mail:  schvan@crl.nmsu.edu.  You  may 
also  write  any  comments  below  or  on  the  back. 

Thank  you  very  much  for  your  assistance. 
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Priority  Rating  Form 


Information  Element 

Takeoff 

Climb 

Transition 
to  Cruise 

Cruise 

Descent 

Approach 

■I 

Aircraft  configuration 

Engine  health 

Fuel  quantity 

Fuel  selection 

RPM  (power) 

Altitude  »  AGL 

Altitude  -  MSL 

Distance 

Bank  angle 

Pitch  (attitude) 

Yaw 

Course 

Heading 

Track 

Waypoints 

Airspeed  ? 

Ground  speed 

Vertical  Velocity 

Time  -  ETA/ETE 

Airport  configuration 

Runway  aim  point 

Runway  remaining 

ATC  Communications 

Traffic  /  other  Comm 

Obstructions 

Traffic 

Airspace 

General  Weather 

Wind 
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APPENDIX  C 

Procedure  for  Collecting  Relatedness  Ratings 

Collection  of  the  relatedness  data  occurred  over  an  eight-month  period.  The  participants  completed  a 
demographic  questionnaire  (see  Appendix  A).  After  the  demographic  form  was  completed,  the  participants  were 
directed  to  the  computer  for  the  relatedness  ratings.  The  first  screen  consisted  of  these  instructions: 

Greetings,  thank  you  for  participating  in  our  study  of  the  information  pilots  need  in  the  course  of  flight.  In 
other  parts  of  the  study,  we  have  determined  how  pilots  view  the  priorities  of  various  elements  of  information 
in  each  phase  of  flight.  Now  we  are  trying  to  characterize  how  pilots  mentally  organize  flight-related 
information.  What  we  need  from  you  in  your  judgments  about  how  closely  related  the  different  elements  of 
information  are  to  each  other. 

We  will  show  you  aU  the  pairs  of  the  elements  listed  below,  and  for  each  pair,  you  should  enter  a  number  from 
1  to  9  indicating  how  related  you  think  the  two  elements  in  each  pair  are.  A  rating  of  9  indicates  that  two 
information  elements  are  very  closely  related,  and  a  rating  of  1  indicates  that  two  information  elements  are  not 
very  related  at  all.  Feel  free  to  ask  if  you  have  any  questions. 

The  second  screen  gave  a  list  of  the  22  information  elements  (see  Table  10). 

The  appearance  of  the  rating  task  is  shown  in  Figure  16.  Ratings  were  selected  by  pressing  a  number  key  (1  to 
9);  the  choice  could  be  changed  until  the  space  bar  was  pressed  when  the  next  pair  would  be  presented.  The  231 
pairs  were  presented  in  a  different  random  order  for  each  of  the  participants. 


1  2  3  4  5  6  7 

8  9 

Unrelated 

Related 

Aircraft  Configuration 

Engine  Health 

Enter  1  to  9  Followed  by  <space> 

231  ratings  to  go 

Figure  16.  The  Relatedness  Rating  Screen 


The  final  screen  presented  a  short  debriefing: 

That’s  it.  Thanks  again  for  your  help.  We  hope  to  contribute  to  improving  aviation  with  this  work.  We 
appreciate  help  from  pilots  such  as  yourself  to  accomplish  our  objectives. 


Cl 


